Papers by Gabrielle De Micheli
CARVQ: Corrective Adaptor with Group Residual Vector Quantization for LLM Embedding Compression (2025.findings-emnlp)
Copied to clipboard
Dayin Gou, Sanghyun Byun, Nilesh Malpeddi, Gabrielle De Micheli, Prathamesh Vaste, Jacob Song, Woo Seong Chung
| Challenge: | Large Language Models typically rely on a large number of parameters for token embedding, leading to substantial storage requirements and memory footprints. |
| Approach: | They propose a corrective Adaptor with group Residual Vector Quantization that can be used to compress the embedding layer without requiring specialized hardware. |
| Outcome: | The proposed corrective adaptor can achieve lower average bitwidth-per-parameter while maintaining reasonable perplexity and accuracy compared to scalar quantization. |